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DESCRIPTION 



IMAGE PICKUP APPARATUS, IMAGE OUTPUT METHOD FOR IMAGE PICKUP 



Technical Field 

The present invention relates to an image pickup 
apparatus and, more particularly, to an image pickup 



Background Art 

Image pickup apparatuses such as video cameras, digital 

video cameras or still video cameras having image pickup 

devices such as CCDs (Charge Couple Devices) are widely and 
15 generally used. 

Image pickup devices such as CCDs are narrow in dynamic 

range compared to silver halide cameras. For this reason, 

when an image is taken against backlight with such an image 

pickup device, lost highlight detail which causes a bright 
20 section to lose grayscale or lost shadow detail which causes 

a dark section to lose grayscale occur in a reproduced image. 

In a conventional type of image pickup apparatus, the 

amount of exposure is adjusted to become correct for a subject, 

by an automatic exposure function, but there still remain many 
25 cases where although a main section of a subject has correct 

exposure, lost highlight detail and the like occurs in a 

background and the like. 

In addition, in the conventional type of image pickup 

apparatus, back compensation has been performed in such a 
30 manner that the reproducibility of an image at a low luminance 

signal level/a middle luminance signal level (a low-middle 
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apparatus having an expandable dynamic range . 
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luminance signal level) is weighted while the detail of a high 
luminance section is lost. However, there are also cases 
where lost highlight details in a background desire to be made 
visible . 

5 For this reason, in order to cope with an image of wide 

dynamic range containing a bright section and a dark section 
as mentioned above, there has been provided an image pickup 
apparatus (wide dynamic range camera) capable of taking an 
image of wide dynamic range by synthesizing an image whose 

10 main subject is a comparatively bright section captured by 
a high speed shutter or the like and an image whose main subject 
is a comparatively dark section captured by a low speed 
shutter or the like. 

However, in the above-mentioned image pickup apparatus 

15 capable of taking an image of wide dynamic range, such as a 
wide dynamic range camera, the proportion in which the dynamic 
range of an output image reproducible as a video signal is 
assigned to a high luminance section in which the luminance 
signal levels correspond to high luminance and to a low-middle 

20 luminance section in which the luminance signal levels 
correspond to low luminance/middle luminance is constantly 
fixed irrespective of the kind of subject. 

Accordingly, for example, if an image signal of the high 
luminance section is absent in the output image, an image 

25 signal assigned to the high luminance section in the dynamic 
range of the output image is absent, while the remaining 
low-middle luminance section only is assigned to the dynamic 
range. This hinders effective use of the dynamic range and 
provides a generally dark image. 

30 The present invention has been conceived in view of the 

above-mentioned problems, and an object of the present 
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invention is to provide a novel and improved image pickup 
apparatus capable of performing dynamic assignment in the 
dynamic range of an output image according to the kind of 
subject . 

5 

Disclosure of the Invention 

To solve the above-mentioned problems, according to a 
first aspect of the present invention, an image pickup 
apparatus may include an image pickup device for picking up 

10 an image of a subject, a signal processing section for 
generating a composite image having a relatively wider 
dynamic range than at least either of the dynamic ranges of 
a long-time exposure image picked up with a relatively long 
exposure time by the image pickup device and a short -time 

15 exposure image picked up with a relatively short exposure time 
by the image pickup device, by synthesizing the long-time 
exposure image and the short -time exposure image, and a 
control section for compressing the composite image and 
dynamically varies the proportion of a high luminance dynamic 

20 range to a low-middle luminance dynamic range in a dynamic 
range of an output image to be outputted as a video signal . 

According to the present invention, a long-time 
exposure image and a short -time exposure image are 
synthesized on the basis of an image signal picked up by the 

25 image pickup device, whereby a composite image of expanded 
dynamic range is generated. Furthermore, the proportion in 
which the dynamic range of the composite image is assigned 
to either the high luminance dynamic range or the low-middle 
luminance dynamic range of an output image is dynamically 

30 varied to compress and assign the high luminance dynamic range 
and the low-middle luminance dynamic range of the composite 
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image. In this construction, for example, even if a 
picked-up image is a dark image in which a signal amount 
occupying its high luminance region is small as a whole, the 
proportion is dynamically varied to an appropriate assignment 
5 proportion, whereby clear images may be constantly output ted 
with correct grayscale. The assignment proportion means the 
proportion in which the dynamic range is assigned to either 
the high luminance dynamic range or the low-middle luminance 
dynamic range . 

10 The control section dynamically varies the assignment 

proportion of the high luminance dynamic range to the 
low-middle luminance dynamic range according to at least a 
luminance region which occupies the composite image . In this 
construction, the assignment proportion may be varied 

15 according to a variation in the degree of brightness which 
occupies the composite image. Accordingly, for example, 
even if a high luminance region in the luminance region 
decreases and a dark image occurs, correct grayscale may be 
obtained by increasing the proportion of assignment to the 

20 low-middle luminance dynamic range with respect to the 
proportion of assignment to the high luminance dynamic range. 

The control section corrects the assignment proportion 
of the high luminance dynamic range to the low-middle 
luminance dynamic range each time the composite image is 

25 generated. In this construction, each time the composite 
image is generated, it is determined whether the assignment 
proportion of the dynamic ranges is appropriate, and if 
inappropriate, the assignment proportion is corrected. 
Accordingly, it is possible to constantly output an image 

30 reproduced with appropriate grayscale. 

The luminance region is at least either a high luminance 
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region or a low-middle luminance region. In this 
construction, it is possible to realize assignment of the 
dynamic ranges by taking note of the high luminance region, 
the low-middle luminance region or the high luminance region 
5 • the low-middle luminance region. In addition, the 
luminance region is not limited to the above-mentioned one, 
and may also be made of, for example, a high luminance region, 
a middle luminance region and a low luminance region. 

The control section dynamically varies the assignment 

10 proportion of the high luminance dynamic range to the 
low-middle luminance dynamic range according to at least an 
average luminance signal level of the high luminance region 
which occupies the composite image. In this construction, 
the assignment proportion is dynamically varied according to 

15 variations in a bright section (the high luminance region) 
of the composite image . Accordingly, it is possible to weight 
the low-middle luminance region which is contained in many 
subjects, and increase the low-middle luminance dynamic 
range . 

20 The control section may be adapted to dynamically vary 

the assignment proportion of the high luminance dynamic range 
to the low-middle luminance dynamic range according to at 
least an average luminance signal level of the low-middle 
luminance region which occupies the composite image. 

25 The control section dynamically varies the assignment 

proportion of the high luminance dynamic range to the 
low-middle luminance dynamic range according to at least the 
high luminance region which occupies the composite image . In 
this construction, the assignment proportion is dynamically 

30 varied according to variations in a bright section (the high 
luminance region) of the composite image. For example, if 
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a bright section decreases, the grayscale of a dark section 
is increased so that the reproducibility thereof may be 
improved . 

The control section may be adapted to dynamically vary 
5 the assignment proportion of the high luminance dynamic range 
to the low-middle luminance dynamic range according to at 
least the low-middle luminance region which occupies the 
composite image - 

The control section at least monotonically varies the 
10 assignment proportion of the high luminance dynamic range to 
the low-middle luminance dynamic range. This construction 
makes it possible to gradually vary the assignment proportion. 
Accordingly, since a sharp variation ' does not occur, the 
variation of the assignment proportion may be applied in many 
15 kinds of locations. The present invention is not limited to 
the above-mentioned example, and may also be carried out in 
the case where the assignment proportion is varied like a 
quadrat i c curve . 

The luminance region may be configured to be at either 
20 a high luminance region or a low-middle luminance region. 

Among pixels constructed in the composite image, pixels 
corresponding to at least a higher luminance signal level than 
a switch luminance signal level may be acquired from the 
short- time exposure image, while pixels corresponding to a 
25 lower luminance signal level than the switch luminance signal 
level may be acquired from the long-time exposure image. 

Among pixels constructed in the composite image, pixels 
corresponding to at least a higher luminance signal level than 
a switch luminance signal level may be acquired from the 
30 long-time exposure image, while pixels corresponding to a 
lower luminance signal level than the switch luminance signal 



6 



S03P1044WO00 



level may be acquired from the short- time exposure image. 

The dynamic range is at least either a high luminance 
dynamic range or a low-middle luminance dynamic range . 

The present invention is characterized in that a 
5 compression gain for compressing a luminance signal level of 
the composite image is determined on the basis of at least 
the assignment proportion of a high luminance dynamic range 
of an output image to a low-middle luminance dynamic range 
thereof . 

10 The present invention may be constructed to determine 

a high luminance compression gain for compressing a luminance 
signal level of the high luminance region in the composite 
image and a low-middle luminance compression gain for 
compressing a luminance signal level of the low-middle 

15 luminance region in the composite image. 

The present invention may be constructed to determine 
for each luminance signal level of the composite image at 
least either a final high luminance compression gain or a 
final low-middle luminance compression gain which is to be 

20 used for compression, on the basis of at least either the high 
luminance compression gain or the low-middle luminance 
compression gain. 

The present invention may be constructed to dynamically 
vary the assignment proportion of the high luminance dynamic 

25 range to the low-middle luminance dynamic range of the output 
image according to at least the high luminance region which 
occupies the composite image . 

Brief Description of the Drawings 
30 Fig. 1 is a perspective view showing a schematic 

construction of a wide dynamic range camera according to the 
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present embodiment . 

Fig. 2 is an explanatory view showing a schematic 
subject in a case in which the interior a house having a window 
according to the present embodiment . 
5 Fig. 3 is a block diagram showing a schematic 

construction of the wide dynamic range camera according to 
the present embodiment - 

Fig. 4 is a block diagram showing a schematic 
construction of a signal preprocessing block according to the 
10 present embodiment . 

Figs. 5A to 5C are explanatory views schematically 
showing the synchronization processing of a timing adjustment 
section according to the present embodiment. 

Fig. 6 is a flowchart schematically showing the image 
15 output processing of an image pickup apparatus according to 
the present embodiment . 

Fig. 7 is an explanatory view schematically showing the 
input/output characteristics of images during synthesis 
processing according to the present embodiment. 
20 Fig. 8 is a cumulative histogram schematically showing 

a distribution of the luminance signal levels of a long-time 
exposure image according to the present embodiment . 

Fig. 9 is a cumulative histogram schematically showing 
a distribution of the luminance signal levels of a short -time 
25 exposure image according to the present embodiment . 

Fig. 10 is an explanatory view schematically showing 
the input/output characteristic of an output image according 
to the present embodiment . 

Fig. 11 is an explanatory view schematically showing 
30 the input/output characteristic of an output image according 
to the present embodiment . 
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Fig. 12 is an explanatory view schematically showing 
a correction function based on the area of a high luminance 
region according to the present embodiment . 

Fig. 13 is an explanatory view schematically showing 
5 a correction function based on an average luminance signal 
level of a high luminance region according to the present 
embodiment . 

Best Modes for Carrying Out the Invention 

10 A preferred embodiment of the present invention will 

be described below in detail with reference to the 
accompanying drawings. In the following description and 
accompanying drawings, constituent elements having 
approximately identical functions are denoted by identical 

15 reference numerals, and repetition of the same description 
is omitted herein. 
(1. Wide Dynamic Range Camera) 

First, a wide dynamic range camera (wide D camera) 
according to the present embodiment will be described with 

20 reference to Figs. 1 and 2. Fig. 1 is a perspective view 
showing a schematic construction of the wide dynamic range 
camera according to the present embodiment. Fig. 2 is an 
explanatory view showing a schematic subject in a house which 
has a window and whose indoor state is being captured 

25 according to the present embodiment . An image pickup 
apparatus 102 according to the present embodiment is capable 
of picking up a still image and/or a moving image. 

As shown in Fig. 2, when the image pickup apparatus 
which is generally used as a digital video camera, a still 

30 video camera and the like is to pick up an indoor image of 
a house having a window 122 in, for example, a daytime period 
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in fine weather, if the image pickup apparatus sets an 
exposure reference for a subject to the person 120 who is 
present indoors, the window 122 which is brighter than the 
person 120 loses grayscales and suffers loss of highlight 
5 detail . 

This is because the image pickup apparatus may not 
handle a dynamic range which extends over a wide range from 
the luminance of the person 12 0 which is relatively smaller 
than that of the window 122 to the luminance of the window 

10 122 which is relatively brighter than that of the person 120. 

To solve the above problem, there is an image pickup 
apparatus called a wide dynamic range camera (wide D camera) . 
The image pickup apparatus has, for example, the function of 
performing processing such as varying the shutter speed of 

15 an electronic shutter and synthesizing a plurality of images 
each having a different exposure time, so that even if an 
exposure reference for a subject is set to the person 120 as 
shown in Fig. 2 , the window 122 may be prevented from suffering 
loss of highlight detail and reproduced as a good grayscale 

20 image . 

As techniques for picking up an image of a subject 
having a wide dynamic range to be reproduced as an output image 
containing various sections from a bright section to a dark 
section, there are, in addition to the above-mentioned method 

25 of synthesizing a bright image and a dark image each having 
a different exposure time, a method of changing the 
sensitivity of an image pickup device in units of pixels, 
extracting only signals having the same exposure condition 
from the image pickup device, reproducing images, and 

30 synthesizing one or not less than two images each having a 
different exposure condition, and a method of dividing 
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incident light by means of a prism and synthesizing a signal 
outputted from an image pickup device and a signal outputted 
from an image pickup device to which is stuck a device having 
a light reduction function like an ND filter (Neutral Density 
5 Filter; a light amount adjustment filter) which reduces 
transmitted light over all wavelengths , i.e., equally reduces 
the amount of incident light - 

The wide D camera 102 shown in Fig. 1 has a dynamic range 
far wider than those of generally used video cameras, and is 

10 capable of picking up an image of a subject having a wide 
dynamic range to be reproduced as an output image containing 
various sections from a bright section to a dark section. 
Accordingly, the wide D camera 102 is suitable for picking 
up an image at an indoor location which is shone with strong 

15 external light, a location exposed to greatly different light 
intensities, and the like. 

For example, the wide D camera 102 is particularly used 
as a surveillance camera or the like which often picks up 
images in the case where the dynamic range greatly varies 

20 between time periods when image pickup is to be performed, 
such as the daytime and the nighttime. In addition, the wide 
D camera 102 used in the present embodiment is not limited 
to the surveillance camera. 

(2. Construction of Each Component of Wide Dynamic Range 
25 Camera) 

The construction of each component of the wide D camera 
102 according to the present embodiment will be described 
below with reference to Fig. 3. Fig. 3 is a block diagram 
showing a schematic construction of the wide dynamic range 
30 camera according to the present embodiment. In Fig. 3, for 
example, a microcomputer 137 corresponds to a control section 
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of the present invention, but is not limited to this example. 

As shown in Fig. 3, the wide D camera 102 according to 
the present embodiment may include an optical component 130, 
an image pickup device 131, a preprocessing section 132, a 
5 signal processing section 133, an output section 134, a TG 
135, an EVR 136, the microcomputer 137 (hereinafter referred 
to as the microcomputer 137) , an optical -component control 
circuit 138, and a recording medium 13 9. 

The optical component 13 0 is an optical filter which 

10 reduces unnecessary wavelengths, such as an ND filter. Light 
incident from a subject onto the optical component 13 0 and 
transmitted therethrough is picked up by the image pickup 
device 131, and is converted into an electrical signal. In 
addition, the optical component 13 0 has, for example, a lens 

15 iris and the like to be controlled by the microcomputer 137 
via the optical -component control circuit 138. The 
microcomputer 137 will be descried later in detail. 

The image pickup device (image pickup device) 131 is 
capable of photoelectrically converting an optical image 

20 received from the subject and outputting an electrical image 
signal, by means of a plurality of pixels respectively made 
of photoelectric conversion devices disposed 
two-dimensionally on a light receiving surface. For example, 
the image pickup device 131 is a solid-state image pickup 

25 device such as a CCD made of a multiplicity of kinds. 

The preprocessing section 132, if the image pickup 
device 131 is a CCD, receives the above-mentioned image signal 
outputted from the image pickup device 131, and then takes 
out a video signal while reducing noise (noise) by the 

30 processing of CDS (a correlated double sampling circuit; 
correlated double sampling) . 
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Furthermore, the preprocessing section 132 gives a gain 
to the video signal to amplify the video signal to a suitable 
signal level as occasion demands, and transmits the video 
signal to the signal processing section 133 . 
5 The main noise of the CCD which constitutes the image 

pickup device 131 may include, for example, transfer noise, 
noise of an output amplifier, reset noise, noise caused by 
dark current, and light shot noise. 

Among the above-mentioned kinds of noise, the transfer 

10 noise is noise produced when charges are being transferred, 
but rarely becomes a problem since buried channel CCDs are 
generally used. The noise of the output amplifier is noise 
of a source follower, and if this noise is reduced, the noise 
of the CCD is ameliorated. 

15 The reset noise is noise produced when the FD (floating 

diffusion) of the CCD is reset, and CDS makes it possible to 
reduce this reset noise. 

The signal processing section 133 is made of two major 
blocks. One is a signal preprocessing block 140 which 

20 performs extraction of an inputted long-time exposure image 
and an inputted short -time exposure image each having a 
different exposure time, correction of the color balance 
between the long-time exposure image and the short -time 
exposure image, and synthesis and compression of the 

25 long-time exposure image and the short-time exposure image. 
The exposure times of the long-time exposure image and the 
short -time exposure image may be individually set according 
to subjects whose images are to be picked up. The signal 
preprocessing block 140 will be described later in detail. 

30 The other block following the signal preprocessing 

block is a signal postprocesing block (not shown) which 
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performs color processing such as aperture (aperture) 
processing for performing y (gamma) correction and contour 
correction, and white balance for adjusting color temperature 
on the basis of ""'white'' contained in the subject. 
5 Furthermore, the signal processing section 133 has at 

least a detection section 143 and is capable of detecting 
level information such as a luminance signal level, color 
information or the like of an image signal inputted at the 
present point in time. The signal processing section 133 is 

10 capable of calculating gains such as compression gains and 
adjusting white balance on the basis of the detected 
information. The detection section 143 will be described 
later in detail . 

The output section 134 receives the video signal from 

15 the signal processing section 133 after the processing of the 
signal processing section 133, and outputs a video image to 
a monitor such as a display device through a driver of the 
output section 134 in order to reproduce the video image. 

The TG (timing generator) 135 generates pulses 

20 necessary for the image pickup device 131 made of a CCD or 
the like. For example, pulses such as 4 -phase pulses for 
vertical transfer, field shift pulses, and 2-phase pulses for 
horizontal transfer are generated. The image pickup device 
131 may be driven (electronic shutter function) by the TG 135 . 

25 By adjusting the driving of the image pickup device 131, 

it is possible to pick up a long-time exposure image having 
a relatively long exposure time and a short-time exposure 
image having a relatively short exposure time. In the image 
pickup apparatus 102 which is the wide dynamic range camera 

30 according to the present embodiment, image pickup processing 
is performed by the above-mentioned processing, but image 
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pickup processing according to the present embodiment is not 
limited to this example. 

The EVR (electronic volume) 136 is a variable resistor 
whose resistance value may be adjusted by digital signals or 
5 the like, and controls the optical -component control circuit 
13 8 and the like by varying the resistance value. Accordingly, 
iris adjustment of a lens which constitutes the optical 
component 130 may be performed by the EVR 136 and the like. 
In addition, the EVR 136 has storage means such as a memory, 

10 and may hold a varied resistance value when a power source 
is turned off. 

The optical -component control circuit 138 has control 
means (not shown) for performing control such as iris 
adjustment of the optical component 13 0 or switching from one 

15 optical filter to another such as an ND filter among a 
plurality of optical components 130. 

The microcomputer 13 7 controls the processing of each 
of the preprocessing section 132, the signal processing 
section 133, the TG 135, the EVR 136 and the optical -component 

20 control circuit 138 on the basis of, for example, a detection 
result from the above-mentioned detection section 143. In 
addition, the microcomputer 137 determines compression gains 
(compression gains) and the like for synthesizing and 
compressing a long-time exposure image (long) and a 

25 short -time exposure image (short) of the image pickup 
apparatus 102 which is the wide dynamic range camera. The 
microcomputer 137 will be described later in detail. 

Furthermore, the microcomputer 13 7 may control the 
image pickup apparatus 102 by mutually communicating, for 

30 example, control data for controlling the apparatus with an 
apparatus outside of the image pickup apparatus 102 via an 
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""external I/F (external interface)". The ""external I/F" is, 
for example, Ethernet (registered trademark) such as lOBase-T 
or lOBase-2, EIA-232, EIA-485 or the like. 

The storage medium 13 9 is capable of storing, for 
5 example, control data necessary for control of individual 
sections provided in the image pickup apparatus 102, 
adjustment data for adjustment of exposure of subjects, user 
setting data such as color correction and AE which may be 
variously set by individual users who use the image pickup 

10 apparatus 102. 

The storage medium 139 may use, for example, a ROM which 
may store data written thereto, and an EEPROM (Electrically 
Erasble and Programmable ROM) which may. store or erase data 
by electrical erasure. 

15 (2.1 Signal Preprocessing Block 140) 

The signal preprocessing block included in the signal 
processing section 13 3 according to the present embodiment 
will be described below with reference to Fig. 4. Fig. 4 is 
a block diagram showing a schematic construction of the signal 

20 preprocessing block according to the present embodiment. 

In' Fig. 4, for example, the microcomputer 137 
corresponds to the control section of the present invention, 
while a compression gain calculation section 145 corresponds 
to a compression gain calculation section of the present 

25 invention, but this example is not restrictive. Furthermore, 
although the compression gain calculation section 145 is 
provided in the signal preprocessing block 140, the present 
invention is not limited to this example, and may also be 
carried out even in various other cases such as the case where 

30 the compression gain calculation section 145 is provided in 
the microcomputer 137 as the control section, and the case 
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where the compression gain calculation section 14 5 is 
provided in a section other than the signal preprocessing 
block 140. 

As shown in Fig. 4, the signal preprocessing block 14 0 
5 may include a timing adjustment section 141, clamp 
processings 142, a detection section 143, a synthesis section 
144, the compression gain calculation section 145, and a 
compression section 146. 

The timing adjustment section 141 adjusts 

10 (synchronizes) the timings of image signals having different 
exposure times, which are respectively outputted from the 
preprocessing section 132 at different timings, to a 
synchronized timing. Synchronization processing in the 
timing adjustment section 141 will be described below. 

15 First, as shown in Fig. 5A, a long-time exposure image 

signal (long signal) containing a long-time exposure image 
(long) picked up by the image pickup device 131, and a 
short -time exposure image signal (short signal) containing 
a short -time exposure image (short) picked up by the same are 

20 transmitted from the preprocessing section 132 to the timing 
adjustment section 141 via one bus (path) shown in Fig. 4 in 
such a manner that the long signal and the short signal are 
alternately arranged by time sharing. Figs. 5A to 5C are 
explanatory views schematically showing the synchronization 

25 processing of the timing adjustment section 141 according to 
the present embodiment . 

When the long-time exposure image signal and the 
short-time exposure image signal are alternately inputted to 
the timing adjustment section 141 in a time-shared manner, 

30 the timing adjustment section 141 performs synchronization 
on the long-time exposure image and the signal short -time 
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exposure image signal. 

The above-mentioned synchronization extracts the 
long-time exposure image and the signal short -time exposure 
image signal contained in one signal, as shown in Fig. 5A, 
5 and adjusts (performs synchronization on) the temporal timing 
of the long-time exposure image contained in the long-time 
exposure image signal and the temporal timing of the 
short -time exposure image contained in the short -time 
exposure image signal, as shown in Fig. 5B (long-time exposure 
10 image signal) and 5C (short-time exposure image signal) . As 
shown in Fig. 4, the timing adjustment section 141 separately 
outputs the short-time exposure image and the long-time 
exposure image to the respective clamp processing sections 
142 . 

15 Since the synchronization is performed in the timing 

adjustment section 141, switching of pixels may be smoothly 
performed when the long-time exposure image and the 
short-time exposure image are to be synthesized in the 
synthesis section 144, whereby the synthesis section 144 may 

20 efficiently carry out synthesis processing. 

Then, the clamp processing sections 142 determine a ^^0 
level (black level)" reference from the luminance signal 
levels of the long-time exposure image and the short-time 
exposure image (clamp processing) . After the determination 

25 of the 0 level, each of the clamp processing sections 142 
outputs the corresponding one of the long-time exposure image 
and the short -time exposure image to both the synthesis 
section 144 and the detection section 143. 

The respective clamp processing sections 142 perform 

30 reproduction of direct current components, reduce low 
frequency noises, and determine where the 0 level is, by 
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making use of the fact that the long-time exposure image 
signal and the short-time exposure image signal have 
periodicity. In composite video signals and luminance 
signals, black levels are used as references, and direct 
5 current values represent information. Accordingly, in 
signal processing, the black levels are fixed, and the 
processing is performed on the basis of these levels. 

The detection section 143 detects what signal amount 
is inputted and what the luminance signal level is, as to each 

10 of the long-time exposure image signal and the short -time 
exposure image signal outputted from the clamp processing 
sections 142. The luminance signal level (luminance level) 
indicates the degree of brightness. 

Namely, when a detection result of the detection 

15 section 143 is outputted to individual sections such as the 
microcomputer 13 7, the individual sections become able to 
execute processing. The detection section 143 detects the 
signal amount or the distribution of the luminance signal 
level (a histogram of the luminance signal level) of each of 

20 the inputted long-time and short -time exposure images so that 
the microcomputer 137 in particular may determine a switch 
point for synthesis of the long-time exposure image and the 
short - 1 ime exposure image . 

The synthesis section 144 performs level adjustment on 

25 the exposure levels of the inputted long-time exposure image 
and short-time exposure image, and synthesizes both exposure 
images into one image by performing switch processing in units 
of pixels. The generated composite image is outputted to the 
compression section 146 and the compression gain calculation 

30 section 145 as a composite image signal. The synthesis 
processing of the synthesis section 144 to synthesize the 
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long-time exposure image and the short -time exposure image 
will be described later in detail. 

The microcomputer 13 7 may receive the signal amounts 
or the histograms of the luminance signal levels detected from 
5 the respective long-time and short-time exposure image 
signals. In addition, an average luminance signal level and 
the like of a high luminance region or a low-middle luminance 
region may be found from the signal amount of each of the 
luminance signal levels. 

10 The microcomputer 137, on the basis of information such 

as the detection result acquired from the detection section 
143, performs processing such as adjustment of the exposures 
of the short -time exposure image and the long-time exposure 
image, correction of the proportion of a high luminance 

15 dynamic range and a low-middle luminance dynamic range in the 
dynamic range of an output image to be outputted as a video 
signal, determination of the switch point for synthesis of 
the short -time exposure image and the short -time exposure 
image, or determination of the compression gains (compression 

20 gains) of the generated composite image. The 
above-mentioned switch point or compression gains and the 
like are transmitted to each of the processing sections of 
the signal preprocessing block 140 by the microcomputer 137. 
In addition, the respective compression gains are constructed 

25 for different luminance regions. The compression gains in 
the present embodiment include a high luminance compression 
gain (Gs) and a low-middle luminance compression gain (Gl) , 
but this example is not restrictive. For example, the 
compression gains may be made of a high luminance compression 

30 gain, a low-middle luminance compression gain and a low 
luminance compression gain. The operation processing of the 
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microcomputer 137, such as determination of the switch point, 
correction of the proportion of the dynamic ranges and 
determination of the compression gains will be described 
later in detail. 
5 During compression of the generated composite image, 

the compression gain calculation section (Compression Gain 
Calculator) 145 calculates final compression gains for the 
respective luminance signal levels to be compressed, on the 
basis of the above-mentioned compression gains (the high 

10 luminance compression gain (Gs) and the low-middle luminance 
compression gain (Gl) ) transmitted from the microcomputer 137, 
and transmits the final compression gains to the compression 
section 146. The final compression gains in the present 
embodiment include a final high luminance compression gain 

15 and a final low-middle luminance compression gain, but this 
example is not restrictive. 

The compression section 14 6 compresses the dynamic 
range of the composite image synthesized by the synthesis 
section 144 to the dynamic range of an output image to be 

20 outputted as a video signal, on the basis of the inputted final 
compression gains. The compressed output image is 
transmitted to the next signal postprocessing block (not 
shown) . 

The output image outputted from the compression section 
25 146 is inputted to the signal postprocessing block, in which 
color processing such as gamma correction, aperture or white 
balance is executed. 

In the color processing, since the gamma characteristic 
of, for example, a CRT (Cathode Ray Tube, a picture tube) is 
30 determined, the image pickup apparatus 102 must perform 
correction in advance so that a reproduced picture may obtain 
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correct grayscale characteristics. Accordingly, the 
correction needs to be performed by gamma correction. In 
general, diode characteristics are used. 

(3. Image Output Operation of Image Pickup Apparatus 102) 
5 An embodiment of the image output operation of the image 

pickup apparatus 102 constructed in the above-mentioned 
manner will be described below. 

(3.1 Image Processing (S160) of Image Pickup Apparatus) 

First, the image pickup processing of the image pickup 

10 apparatus 102 will be described below with reference to Figs. 
3, 4 and 6. Fig. 6 is a flowchart schematically showing the 
image output processing of the image pickup apparatus 
according to the present embodiment . 

First, as shown in Fig. 6, the image pickup apparatus 

15 102 which is a wide dynamic camera performs image pickup 
processing (S160) of a subject by means of the image pickup 
device 131. Although the image pickup processing according 
to the present embodiment is image pickup processing based 
on a double exposure method, the present invention is not 

20 limited to this example, and may also be carried out in the 
case of, for example, a 2 -CCD type which uses two image pickup 
devices 131 such as CCDs and has a standard- speed shutter and 
a high-speed shutter. 

The double exposure method is a method of picking up 

25 a long-time exposure image of a subject which needs a 
relatively long exposure time and whose correct exposure is 
in a low-middle luminance region and a short -time exposure 
image of a subject whose correct exposure is in a high 
luminance region, and synthesizing both images to expand an 

30 apparent dynamic range. The term luminance region" means 
a region formed by an aggregation of pixels or signals 
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constructed in an image having approximately the same 
luminance level. For example, a high luminance region 
denotes a region formed by pixels or signals of high luminance 
levels, among pixels or signals constructed in an image. 
5 After the above-mentioned image pickup processing 

(316 0) , the long-time exposure image (long) and the 
short-time exposure image (short) which have been outputted 
from the image pickup device 131 and subjected to, for example, 
CDS processing by the preprocessing section 132 are 

10 transmitted to the timing adjustment section 141 of the signal 
preprocessing block 140 of the signal processing section 133 
shovm in Fig . 4 . 

The above-mentioned both image signals are alternately 
inputted to the timing adjustment section 141 via one bus at 

15 different timings in such a manner that the long-time exposure 
image (long) and the short -time exposure image (short) are 
time-shared, so that both image signals need to be 
synchronized as described above. 

(3.2. Detection Processing (S162) of Image Pickup Apparatus 
20 102) 

After the long-time exposure image and the short -time 
exposure image have been synchronized in the timing 
adjustment section 141, both exposure images are separately 
outputted from the timing adjustment section 141 and inputted 

25 to the clamp processing sections 142. After having been 
subjected to clamp-processing in the clamp processing 
sections 142, the long-time exposure image and the short -time 
exposure image are transmitted to each of the detection 
section 143 and the synthesis section 144 . 

30 When the long-time exposure image and the short -time 

exposure image are transmitted to the synthesis section 144, 
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the synthesis section 144 performs detection processing 
(S162) on the long-time exposure image and the short -time 
exposure image . 

The synthesis section 144 detects through the detection 
5 processing (S162) of the long-time exposure image and the 
short -time exposure image, for example, signal amounts at the 
respective signal levels of the long-time exposure image, 
signal amounts at the respective signal levels of the 
short -time exposure image, a distribution (histogram) of the 

10 signal amounts at the respective signal levels of the 
long-time exposure image, or a distribution of the signal 
amounts at the respective signal levels of the short -time 
exposure image . 

On the basis of the detection result in the detection 

15 processing (S162), the microcomputer 137 determines La 
(switch luminance signal level) indicative of the highest 
luminance signal level in the low-middle luminance region in 
the long-time exposure image, and Lb indicative of the highest 
luminance signal level in the high luminance region in the 

20 short -time exposure image. La and Lb will be described later 
in detail. 

(3.3. Synthesis Processing (S164) of Image Pickup Apparatus 
102) 

After the detection processing (S162) , the synthesis 
25 processing (S164) is performed on the long-time exposure 
image and the short -time exposure image in the synthesis 
section 144, whereby a composite image is generated. The 
synthesis section 144 performs the synthesis processing 
(S164) on the basis of information transmitted from the 
30 microcomputer 137, for example, La. The synthesis 
processing will be described below in detail. 
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The synthesis processing according to the present 
embodiment performs switching on the basis of the switch point 
(Switch point) so as to adopt pixels corresponding to the 
luminance signal levels of the long-time exposure image as 
5 to luminance signal levels lower than the switch point and, 
when the switch point is exceeded, adopt pixels corresponding 
to the luminance signal levels of the short-time exposure 
image . 

Accordingly, one composite image is synthesized by 

10 switching a target to be adopted, from the long-time exposure 
image to the short-time exposure image in units of pixels. 
The switch point according to the present embodiment will be 
described below with reference to Fig. 7. Fig. 7 is an 
explanatory view schematically showing the input/output 

15 characteristics of images during the synthesis processing 
according to the present embodiment . 

As shown in Fig. 7, the input /output characteristic of 
the short -time exposure image (short) is a short -time 
input/output characteristic 170, the input/output 

20 characteristic of the long-time exposure image (long) is a 
long-time input/output characteristic 171, and the 
input /output characteristic of the output image output ted 
from the signal processing section 133 as the video signal 
is an output image input/output characteristic 172. The 

25 horizontal axis shown in Fig. 7 represents the luminance 
signal levels of image signals inputted to the synthesis 
section 144, while the vertical axis represents the luminance 
signal levels of image signals outputted from the synthesis 
section 144. 

30 First, for synthesis, the exposure ratio of the 

long-time exposure image to the short-time exposure image is 
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multiplied by the short -time exposure image, whereby the 
levels of both images are adjusted. For example, if the 
exposure ratio of the long-time exposure image to the 
short -time exposure image is 10:1, the exposure of the 
5 short-time exposure image is one-tenth of the long-time 
exposure image. However, the amount of existing light is ten 
times the luminance signal levels of the short -time exposure 
image. Accordingly, the levels are adjusted by multiplying 
the short -time exposure image by 10. 

10 Accordingly, the inclination of the short-time 

input/output characteristic 170 moves in the direction of the 
arrow shown in Fig. 7, and the level of the short- time 
input /output characteristic 170 coincides with that of the 
long-time input/output characteristic 171. Furthermore, on 

15 the basis of an appropriate switch point (switch point) , the 
short-time input/output characteristic 170 is inclined by the 
amount of predetermined inclination, whereby the output image 
input /output characteristic 172 is obtained. 

The predetermined inclination is stored in, for example, 

20 the storage medium 13 9, and further, the microcomputer 137 
performs the processing of inclining the short-time 
input/output characteristic 171 level-adjusted by the 
above-mentioned multiplication, by the amount of the 
predetermined inclination. The reason for the short- time 

25 input/output characteristic 171 is inclined by the 
above-mentioned amount of inclination is that the dynamic 
range is very wide and noise such as image distortion needs 
to be avoided. 

When the level adjustment of the above-mentioned both 

30 images is completed, pixels to be adopted for a composite 
image are switched from the long-time exposure image to the 
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short-time exposure image on the basis of the switch point 
(switch point) shown in Fig. 7, whereby one composite image 
is synthesized. Accordingly, a composite image is generated 
which has the characteristics of the long-time exposure image 
5 in which a dark section corresponding to the low-middle 
luminance signal levels is reproduced with good grayscale, 
and the characteristics of the short -time exposure image in 
which a bright section corresponding to the high luminance 
signal levels is reproduced with good grayscale, whereby the 
10 bright section and the dark section are reproduced with good 
grayscale . 

(3.3.1 Exclusion Processing of Synthesis Section 144) 

In the above-mentioned synthesis processing (S164) , 
the pixels for the composite image are not equally adopted 

15 from the luminance signal levels of both the long-time 
exposure image and the short -time exposure image, and if there 
is not a signal amount corresponding to a luminance signal 
level, the exclusion processing of excluding the luminance 
signal level from the target to be synthesized is performed. 

20 This means that the luminance signal level is not 

assigned to (is excluded from) the dynamic range of the 
composite image. Accordingly, it is possible to make 
effective use of the dynamic range to be assigned to the 
composite image. 

25 The exclusion processing according to the present 

embodiment for making effective use of the dynamic range in 
the synthesis processing (S164) will be described below with 
reference to Figs. 8 and 9. Fig. 8 is a cumulative histogram 
schematically showing a distribution of the luminance signal 

30 levels of the long-time exposure image according to the 
present embodiment, and Fig. 9 is a cumulative histogram 
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schematically showing a distribution of the luminance signal 
levels of the short-time exposure image according to the 
present embodiment . 

First, as shown in Fig. 8, as the luminance signal level 
5 becomes gradually higher toward the highest luminance signal 
level La (switch luminance signal level) in the low-middle 
luminance region, the signal amount is accumulated. When the 
luminance signal level exceeds La, the signal amount is not 
accumulated in the range of luminance signal levels shown as 

10 a range 180. Accordingly, in the range 180, a signal or a 
pixel is absent. In Figs. 8 and 9, the horizontal axis 
represents the luminance signal levels of image signals 
inputted to the synthesis section 144, while the vertical axis 
represents a cumulative signal amount. 

15 If a signal or a pixel corresponding to a luminance 

signal level is absent like in the range 18 0 shown in Fig. 
8, the luminance signal level corresponding to the range 180 
is excluded from the target for the synthesis processing 
(S164) . Accordingly, the excluded luminance signal level is 

20 not assigned to the dynamic range of the composite image, 
whereby it is possible to make effective use of the dynamic 
range . 

Furthermore, although signals or pixels corresponding 
to luminance signal levels in a range 182, shown in Fig. 8, 

25 which is a high luminance region, are present, the luminance 
signal levels overlap the luminance signal levels of the 
short-time exposure image shown in Fig. 9 which will be 
mentioned later. Since the luminance signal levels of the 
high luminance region in the long-time exposure image in 

30 particular overlap the luminance signal levels of the 
short- time exposure image, the luminance signal levels are 
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excluded from the target for the synthesis processing (S164) . 
Although the synthesis processing according to the present 
embodiment has been described with illustrative reference to 
the case where the overlapping luminance signal levels are 
5 excluded, the present invention is not limited to this example, 
and may also be carried out in the case where, for example, 
a luminance signal level is set in advance and excluded. 

Namely, as shown in Fig. 8, the pixels corresponding 
to the luminance signal levels in the long-time exposure image 

10 for the synthesis processing (S164) from which the range 180 
and the range 182 are excluded become a target to be adopted 
for the composite image. Accordingly, the range of luminance 
signal levels to be adopted for the synthesis processing 
(S164) is a range lower than La. Namely, by setting the 

15 above-mentioned La as the switch point (switch point) shown 
in Fig. 7, it is possible to efficiently use necessary pixels 
the long-time exposure image, and it is possible to make 
effective use of the dynamic range of the composite image. 
Although La in the present embodiment is determined by the 

20 microcomputer 13 7, the present invention is not limited to 
this example, and may also be carried out in the case of, for 
example, the detection section 143. 

The luminance regions according to the present 
embodiment are regions each having a predetermined luminance 

25 signal level range in an image, and the region in which 
luminances relatively correspond to low-middle luminances is 
made the low-middle luminance region, while the region in 
which luminances relatively correspond to high luminances is 
made the high luminance region. However, the present 

30 invention is not limited to this example, and may also be 
carried out in the case where, for example, the low-middle 
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luminance region is further divided into a low luminance 

region and a middle luminance region. 

Then, as shown in Fig. 9, the highest luminance signal 

level of the high luminance region in the short -time exposure 
5 image is set to Lb, and pixels corresponding to lower 

luminance signal levels than Lb are made a target to be applied 

to the composite image. 

As described previously, in the short -time exposure 

image as well, if a signal or a pixel corresponding to a 
10 luminance signal level is absent, the luminance signal level 

is excluded from the target for the synthesis processing 

(S164) . Accordingly, the excluded luminance signal level is 

not assigned to the dynamic range of the composite image, 

whereby it is possible to make effective use of the dynamic 
15 range with high use efficiency by assigning a section in which 

grayscales are absent to other luminance signal levels. 

Furthermore, the present embodiment has been described 

with illustrative reference to the case where the switch point 

(switch point ) is La (switch luminance signal level) , but the 
20 present invention is not limited to this example and may also 

be carried out in the case where a luminance signal level which 

is set in advance is used as a switch point . 

(3.4 Compression Processing (S168) in Image Pickup Apparatus 

102) 

25 In the image output operation of the image pickup 

apparatus 102 according to the present embodiment, the next 
processing following the synthesis processing (S164) is 
assignment processing (S166) as shown in Fig. 6, but 
compression processing (S168) will first be described below. 

30 The dynamic range of the composite image synthesized 

in the synthesis processing (S164) is far wider than that of 
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an image of, for example, a generally used type of video 
camera . 

Accordingly, none of the processing sections (devices) 
provided in the image pickup apparatus 102 which is the wide 
5 dynamic range camera is able to process the composite image 
signal, so that the dynamic range of the composite image needs 
to be compressed to a processable dynamic range (the dynamic 
range of the output image) . 

First, the compression processing according to the 

10 present embodiment will be described below with reference to 
Fig. 10. Fig. 10 is an explanatory view schematically showing 
the input/output characteristic of the output image according 
to the present embodiment. In Fig. 10, the horizontal axis 
represents the luminance signal level of an image signal 

15 inputted to the compression section 146, while the vertical 
axis represents the luminance signal level of an image signal 
outputted from the compression section 146. 

As shown in Fig. 10, the input/output characteristic 
of the composite image is a composite input/output 

20 characteristic 202, and the input/output characteristic of 
the output image in which the dynamic range of the composite 
image is compressed is the output image input/output 
characteristic 172. The output image is outputted from the 
signal preprocessing block 14 0 as a video signal, and is 

25 transmitted to the next signal postprocessing block (not 
shown) . 

The proportion of a low-middle luminance dynamic range 
to a high luminance dynamic range in the dynamic range of the 
composite image shown in Fig. 10 is a:b, where b = 1 - a. The 
30 proportion of a low-middle luminance dynamic range to a high 
luminance dynamic range in the dynamic range of the output 
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image is mm, where n = 1 - m. 

The dynamic range (dynamic range) according to the 
present embodiment may include the high luminance dynamic 
range which is the dynamic range of the high luminance region 
5 and the low-middle luminance dynamic range which is the 
dynamic range of the low-middle luminance region. 

The dynamic range is the range of brightness of a subject 
which may be handled by an image pickup device such as an image 
pickup device, i.e., the range of reproducible grayscales. 

10 Accordingly, the brightness or grayscale of an image 

dramatically varies according to what proportion of the 
grayscale of a subject should be assigned to which of the high 
luminance dynamic range and the low-middle luminance dynamic 
range in the above-mentioned dynamic range. For example, if 

15 the grayscale of the high luminance region is to be weighted 
in the dynamic range of the output image, the proportion of 
n in the reproduction ratio of m:n is increased to increase 
the proportion of the high luminance dynamic range, whereby 
the detail of the grayscale of the high luminance region is 

20 reproduced . 

Then, the compression gains for compressing the dynamic 
range of the composite image are found by the microcomputer 
137. The compression gains found by the microcomputer 137 
are the low-middle luminance compression gain (Gl) found on 

25 the basis of La in the low-middle luminance region and the 
high luminance compression gain (Gs) found on the basis of 
Lb in the high luminance region. 

Since the low-middle luminance dynamic range of the 
composite image at La is compressed to the low-middle 

30 luminance dynamic range of the output image as shown in Fig. 
10, the low-middle luminance compression gain (Gl) may be 
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expressed by the following equation: 
Gl = m / La (Eq. 1) 

La or Lb is found by the microcomputer 137 on the basis 
of the detection result by the detection section 143, such 
5 as the signal amounts or the histogram of the luminance signal 
levels . 

Similarly, the high luminance compression gain is found 
by the microcomputer 13 7. Since the range of the high 
luminance region continues to Lb, the high luminance region 
10 between Lb and La needs to be accommodated into n. Therefore, 
the high luminance compression gain (Gs) at Lb may be 
expressed by the following equation: 

Gs = n / (Lb - La) (Eq. 2) 

The high luminance dynamic range and the low-middle 
15 luminance dynamic range of the composite image which are found 
by the equations (1) and (2) are respectively compressed by 
the compression section 146 in the directions indicated by 
the arrows shown in Fig. 10, and the dynamic range of the 
composite image is assigned to the dynamic range of the output 
20 image . 

Namely, it may be said that the assignment proportion 
of the high luminance dynamic range to the low-middle 
luminance dynamic range in the dynamic range of the composite 
image is assigned to the assignment proportion of the high 

25 luminance dynamic range to the low-middle luminance dynamic 
range in the dynamic range of the output image. 

In addition, the low-middle luminance compression gain 
(Gl) and the high luminance compression gain (Gs) which are 
respectively found by the equations (1) and (2) are 

30 compression gains at the input luminance signal levels La and 
Lb, respectively. Accordingly, to be exact, the compression 
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gains at the individual luminance signal levels between La 
and Lb differ from Gl or Gs . 

The final compression gains are found by the 
compression gain calculation section 145. The compression 
5 gain calculation section 145 finds, on the basis of Gl and 
Gs transmitted from the microcomputer 137, a final low-middle 
luminance compression gain at each luminance signal level in 
the low-middle luminance region and a final high luminance 
compression gain at each luminance signal level in the high 
10 luminance region. Then, the compression gain calculation 
section 14 5 transmits the final high luminance compression 
gain and the final low-middle luminance compression gain to 
the compression section 14 6. 

(3.5 Assignment Processing (S166) of Image Pickup Apparatus 
15 102) 

The assignment processing (S166) according to the 
present embodiment will be described below with reference to 
Fig. 11. Fig. 11 is an explanatory view schematically showing 
the input/output characteristic of the output image according 

20 to the present embodiment . 

Approximately like in Fig. 10, the input /output 
characteristic of the output image shown in Fig. 11 is the 
output image input/output characteristic 172, the 
input/output characteristic of a corrected output image is 

25 a corrected output image input /output characteristic 173, and 
the input/output characteristic of the composite image is the 
composite input/output characteristic 202. The proportion 
(reproduction ratio) of the low-middle luminance dynamic 
range to the high luminance dynamic range in the dynamic range 

30 of the output image is m:n, where n = 1 - m. In Fig. 11, the 
horizontal axis represents the luminance signal level of an 
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image signal inputted to the compression section 146, while 
the vertical axis represents the luminance signal level of 
an image signal outputted from the compression section 146. 

For example, when no signals at all exist in the high 
5 luminance region of a composite image, if an output image is 
reproduced by assigning the dynamic range of the composite 
image to only the range ^'m'' which is the low-middle luminance 
dynamic range of the low~middle luminance region in the 
dynamic range of the output image, the output image exhibits 

10 a generally dark grayscale. The above-mentioned assignment 
of the dynamic range may be deemed inappropriate assignment 
and inefficient assignment. 

Accordingly, the microcomputer 137 determines whether 
the proportion of a high luminance dynamic range to a 

15 low-middle luminance dynamic range in the dynamic range of 
an output image outputted previously is appropriate for a 
composite image to be newly inputted, and if the proportion 
is inappropriate, the microcomputer 13 7 performs correction 
by changing the proportion of the high luminance dynamic range 

20 to the low-middle luminance dynamic range to an appropriate 
proportion. Accordingly, it is possible to achieve 
efficient and appropriate assignment of the dynamic range. 

To enable the above-mentioned assignment, the 
assignment proportion of the high luminance dynamic range to 

25 the low-middle luminance dynamic range in the dynamic range 
of the output image is dynamically varied for each input 
composite image. The dynamic change of the proportion 
according to the present embodiment is executed each time a 
composite image is inputted, but the present invention is not 

30 limited to this example and may also be carried out in the 
case where the assignment proportion of the dynamic ranges 
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is changed each time, for example, five composite images are 
inputted . 

In the dynamic change of the assignment proportion of 
the high luminance dynamic range to the low-middle luminance 
5 dynamic range, the proportion of the high luminance dynamic 
range to the low-middle luminance dynamic range, i.e., mm, 
is corrected according to the signal amount of the high 
luminance region which occupies the composite image. 

The correction of the proportion of the high luminance 

10 dynamic range to the low-middle luminance dynamic range 
according to the present embodiment is performed on the basis 
of the proportion of the high luminance region found from the 
signal amount of the high luminance region (the area of the 
high luminance region) or the average luminance signal level 

15 of the high luminance region. The case of the area of the 
high luminance region and the case of the average luminance 
signal level of the high luminance region will be separately 
described below. 

The correction of the dynamic range according to the 

20 present embodiment will be described with illustrative 
reference to the case where the correction is performed 
according to the high luminance region occupying the 
composite image, but the present invention is not limited to 
this example and may also be carried out in the case where 

25 the dynamic range is corrected on the basis of an area or an 
average luminance signal level found from the signal amount 
of the low luminance region or low-middle luminance region. 
(3.5.1 Assignment Processing (S166) Based on Area of High 
Luminance Region) 

30 If m' :n' is used to denote a corrected proportion 

obtained by correcting the proportion (mm) of the low-middle 
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luminance dynamic range to the high luminance dynamic range, 
m' and n' may be expressed by the following equations: 

m* = f (x) xn + m (Eq. 3) 

n' = 1 - m' (Eq. 4) 

5 In the equation (3), f (x) is a correction function based 

on the area of the high luminance region. The input /output 
characteristic of the correction function f (x) will be 
described below with reference to Fig. 12. Fig. 12 is an 
explanatory view schematically showing the correction 

10 function based on the area of the high luminance region. 

As shown in Fig. 12, the input/output characteristic 
of the correction function is an area input/output 
characteristic 220. The horizontal axis represents the 
proportion of the area of the high luminance region in the 

15 composite image, while the vertical axis represents 
correction values of the correction function f (x) . 

As shown in Fig. 12, when the proportion of the area 
of the high luminance region is less than Rb and not less than 
Ra, the correction value of the correction function f (x) takes 

20 any value from ^"1.0" to ""0". Furthermore, m' and n' of the 
corrected proportion are found from the equations (3) and (4) , 
whereby the assignment processing (S166) is performed. If 
the proportion of the area is not less than Ra, the entire 
high luminance dynamic range is assigned to the low-middle 

25 luminance dynamic range in the assignment processing (3166) , 
while if the proportion of the area exceeds Rb, the assignment 
processing (S166) is not performed and the proportion of the 
high luminance dynamic range to the low-middle luminance 
dynamic range remains unchanged. 

30 Accordingly, as shown in Figs. 11 and 12, if the 

proportion of the area of the high luminance region is less 
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than Rb and not less than Ra, as the proportion of the area 
of the high luminance region approaches Ra, m' of the 
corrected proportion increases. Namely, the corrected 
output image input/output characteristic 173 moves on the La 
5 axis in the vertically upward direction . This fact means that 
since the high luminance region decreases, the proportion in 
which the high luminance dynamic range is assigned to the 
low-middle luminance dynamic range increases. In contrast, 
as the proportion of the area of the high luminance region 

10 approaches Rb, m' of the corrected proportion decreases, and 
the corrected output image input /output characteristic shown 
in Fig. 11 moves on the La axis in the vertically downward 
direction, and approaches m. 

In addition, as shown in Fig. 12, if the proportion of 

15 the area of the high luminance region is less than Ra, all 
the high luminance dynamic range of the output image assigned 
to the high luminance region is assigned to the low-middle 
luminance dynamic range . 

Accordingly, as the high luminance region decreases, 

20 the proportion of the dynamic range to be assigned to the 
low-middle luminance dynamic range increases, whereby the 
grayscale reproducibility of the low-middle luminance region 
is improved. Namely, the proportion of the high luminance 
dynamic range to the low-middle luminance dynamic range is 

25 dynamically varied for each inputted output image according 
to the signal amount of the high luminance region thereof, 
whereby the dynamic range is corrected into a dynamic range 
appropriate for the output image . 

Although the above description has referred to the 

30 example in which the correction function f (x) according to 
the present embodiment draws a monotonous variation, the 
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present invention is not limited to this example and may also 
be carried out in the case where the correction function f (x) 
draws, for example, a quadratic curve. 

The assignment processing (S166) based on the average 
5 luminance signal level of the high luminance region has 
approximately the same configuration as the assignment 
processing (S166) based on the area of the high luminance 
region, and as shown in Fig. 13, the input /output 
characteristic based on the average luminance signal level 

10 of the high luminance region assumes an average luminance 
signal level 23 0, and the horizontal axis represents the 
average luminance signal level of the high luminance region. 

Accordingly, as shown in Figs. 11 and 13, if the average 
luminance signal level of the high luminance region is less 

15 than Rb and not less than Ra, as the proportion of the area 
of the high luminance region approaches Ra, m' of the 
corrected proportion increases. Namely, the corrected 
output image input/output characteristic 173 moves on the La 
axis in the vertically upward direction . This fact means that 

20 since the high luminance region decreases, the proportion in 
which the high luminance dynamic range is assigned to the 
low-middle luminance dynamic range increases . 

In addition, as shown in Fig. 13, if the average 
luminance signal level of the high luminance region is less 

25 than Ra, all the high luminance dynamic range of the output 
image assigned to the high luminance region is assigned to 
the low-middle luminance dynamic range. 

Accordingly, as the signal amount of the high luminance 
region decreases, the proportion of the dynamic range to be 

30 assigned to the low-middle luminance dynamic range increases, 
whereby the grayscale reproducibility of the low-middle 
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luminance region is improved. Namely, the proportion of the 
high luminance dynamic range to the low-middle luminance 
dynamic range is dynamically varied for each inputted output 
image according to the signal amount of the high luminance 
5 region thereof, whereby the dynamic range is corrected into 
a dynamic range appropriate for the output image. 
Incidentally, m' and n' of the corrected proportion are found 
by the microcomputer 137. 

In addition, Ra and Rb according to the present 

10 embodiment may be appropriately varied according to the 
characteristics of the image pickup apparatus 102, the kind 
of subject whose image is to be picked up, and the like. 

Furthermore, the present invention may also be carried 
out in the case where both the area of the high luminance region 

15 and the average luminance signal level thereof are used in 
combination. In this case, if x and x' are respectively used 
to denote the proportion of the area of the high luminance 
region and the average luminance signal level of the high 
luminance section, the correction function is expressed as 

20 F(x, x' ) . In addition, if the corrected proportion is made 
m' :n' like in the above description, m' and n' maybe expressed 
by the following equations: 

m' = F(x, X' ) X n + m ( Eq . 5 ) 

n' = 1 - m' (Eq. 6) 

25 Since the equations (5) and (6) have approximately the 

same configuration as the above-described ones, the detailed 
description of the equations (5) and (6) is omitted. 

When the compression processing (S168) is completed, 
the compression section 146 transmits the output image to the 

30 signal postprocessing block as a video signal, and one 
sequence of image output operations is completed. 
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The dynamic range according to the present embodiment 
may include the low-middle luminance dynamic range which is 
the dynamic range of a region having luminances relatively 
corresponding to low-middle luminances, and the high 
5 luminance dynamic range which is the dynamic range of a region 
having luminances relatively corresponding to high 
luminances- However, the present invention is not limited 
to this example and may also be carried out in the case where, 
for example, the low-middle luminance dynamic range is 

10 further divided into a low luminance dynamic range and a 
middle luminance dynamic range . 

Although the preferred embodiment of the present 
invention has been described above with reference to the 
accompanying drawings, the present invention is not limited 

15 to this example. It will be readily apparent to those skilled 
in the art that various changes or modifications may be made 
without departing from the technical idea set forth in the 
appended claims, and all such changes and modifications are, 
of course, considered to fall within the technical scope of 

20 the invention. 

Although the above description of the embodiment has 
referred to the example in which the image pickup apparatus 
102 picks up a black-and-white image, the present invention 
is not limited to this example, and may also be carried out 

25 in the case where the image pickup apparatus 102 picks up, 
for example, a color image or both a color image and a 
black-and-white image . 

Industrial Applicability 
30 As described hereinabove, according to the present 

invention, dynamic ranges respectively assigned to a 
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long-time exposure image and a short-time exposure image are 
dynamically varied according to the area of a high luminance 
region or the average luminance signal level of a high 
luminance section, whereby images may be reproduced with, for 
example, grayscale and exposure constantly maintained under 
optimum conditions. Furthermore, during synthesis, an 
unnecessary section such as the range of luminance signal 
levels in which signals are absent or the range of luminance 
signal levels overlapping the short-time exposure image is 
excluded from the long-time exposure image, whereby it is 
possible to make far more efficient use of the dynamic ranges . 
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